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Family League 2011-12 Out of School Time Programs in Baltimore City 

Executive Summary 

Out of School Time (OST) programs have been shown to promote positive personal, academic 
and social development (Huang, Gribbons, Kim, Lee, & Baker, 2000; Welsh et al., 2002). The 
Family League of Baltimore City works with partners to sponsor a range of after-school 
programs in Baltimore City Public Schools (City Schools) to provide healthy safe environments 
for children after school including nutritious meals. The strategic planning and development of 
the program logic model provides an ultimate outcome of increased graduation rates and career 
and college readiness. More short-term goals include increased student attendance and academic 
performance. Other desired outcomes are improved health and social and emotional 
development, but to date, no measures have been identified to document these changes. This 
report describes the students served and focuses on the outcomes of attendance and academic 
performance. 

OST programs enrolled 3,523 students from kindergarten (K) through grade 12 in 64 Family 
League sponsored OST programs in 48 schools. The OST programs served fewer males than 
females, particularly in the middle grades. They did enroll more students eligible for FARMS 
(92.4%) than the City Schools average (84.5%), yet fewer receiving special education services 
(13.3% vs. 16.5% citywide). 

Outcome analyses focused on students who met a threshold of participation established by 
program staff from the Family League. The threshold was attending a specific number of hours 
that would provide enough exposure to the program to expect a change in behavior. These OST 
students who met the threshold (OST regular attenders) were more likely than the City Schools 
population as a whole to be female, African American, and to receive free/reduced price lunch, 
were less likely to receive special education services, and had higher school attendance the 
previous school year. Students who enrolled in OST programs but did not meet the attendance 
threshold for outcome analysis were more likely to be in middle or high school, and to receive 
special education services. 

OST Regular Attenders 

• Had higher rates of school attendance than similar peers from the same school and this 
pattern continued into the next school year. 

• Had higher promotion rates than their peers, and high school students had a higher rate of 
credit accrual. 

• Entering 6 th and 9 th graders had higher school attendance through the first three quarters of 
2012-13 as they were transitioning into middle grades and high school, respectively. 

• Had significantly fewer chronically absent students in 201 1-12 than comparable peers 
across the district. And for those who were chronically absent in 2010-1 1, more than two- 
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thirds ( 67.7% ) of OST regular attenders were no longer chronically absent while slightly 
less than half (48.0%) of the nonparticipants continued to be CA. 


In sum, while OST participation was associated with higher school attendance and promotion 
rates, this analysis cannot make a causal statement that participation increased attendance or 
promotion. 

As a result from the analysis, we recommend that the Family League: 

• Utilize the Community Schools strategy to better recruit priority students, so that 
services will better reach those students who need and will benefit from them the 
most, especially, students with a history of chronic absenteeism and those who 
receive special education services. 

• Collect and monitor the reasons why students leave or are dismissed from OST 
programs to help OST programs retain students and identify any barriers to student 
attendance. 

• Disaggregate results data by program/school to begin looking at which programs 
are best recruiting priority youth, and which programs are demonstrating the most 
impact. 

• Identify best practices or strong models for serving high school youth and 
implement them in Community School High Schools. 

• Link data for youth served by multiple initiatives, for example, youth in OST who 
use health resources, or youth who were in both an OST and a summer school 
program. 

• Adopt and implement a validated tool like the California Healthy Kids Survey to 
measure social / emotional outcomes for out of school time youth. 
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Family League Out of School Time Programs in Baltimore City 

Linda S. Olson, Faith Connolly, and Alok H. Kommajesula 

Background 

The Family League and its partners have been working for over 10 years to increase the quality 
and quantity of out of school time (OST) programs in Baltimore City. They believe that OST 
programs are very important to the development of Baltimore’s children and youth. Besides 
providing a safe, nurturing environment during the time of day when children are more likely to 
be unsupervised (after school from 3 p.m. - 6 p.m. and weekdays during the summer) they help 
youth develop academically, physically, and socially/emotionally, increasing the likelihood that 
youth will attend and succeed in school, and graduate high school college and career ready. 

Research has identified chronic absenteeism as an early warning indicator for school failure 
(BERC, 201 1). For this reason, the broad Baltimore City community including the Mayor’s 
Office, Baltimore City Schools, the Family Feague and many others, have identified reducing 
chronic absenteeism as a primary goal for improving school success and increasing high school 
graduation. While Baltimore City Schools has made strides to improve attendance and decrease 
chronic absenteeism (see chart below), this remains a critical area of focus. 

Percent of Baltimore City Students Who Were Chronically Absent by 
Grade Span for 2009-10 through 2011-12 


Grade Span 

2011-12 

2010-11 

2009-10 

Elementary(l-5) 

13.2% 

16.3% 

13.9% 

Middle (6-8) 

15.9% 

16.4% 

17.5% 

High (9-12) 

41.3% 

42.2% 

41.9% 


Source: MDreportcard 


Chronic absenteeism may signal family challenges, economic hardship such as homelessness, 
unemployment, poor student, sibling or parent health, student disengagement and other 
obstacles. Research has shown that high quality afterschool programs can improve school-day 
attendance (Huang, Gribbons, Kim, Fee, & Baker, 2000; Welsh et al., 2002). The Family 
Feague has been working with the Baltimore Education Research Consortium for the past three 
years to evaluate the impact of participation in out of school time programs and key youth 
outcomes, including school attendance. 

The Family League of Baltimore City 

The Family Feague is a nonprofit organization that convenes, coordinates and funds programs 
to strengthen the lives of children and families in Baltimore City. Their mission is to improve 
the lives of Baltimore’s children from birth to the time they enter adulthood and begin careers. 
Every day, Family Feague programs touch the lives of thousands of children and families. As 
Baltimore City’s local management board, the Family League occupies a unique role in the city, 
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bringing together public and private dollars and a wide range of partners to fund effective social 
service programs. By using evidence- based approaches, along with analyzing data and 
program results, the goal is to make smarter investments in programs that work, supporting the 
most successful initiatives to best support Baltimore’s families and children. 

The Family League coordinates major initiatives, brings together a range of partners and 
fashions new approaches to the urgent problems facing Baltimore children and families. The 
Family League aims to use state and city funding wisely, leveraging millions in federal and 
private funds to improve the lives of thousands of city children and families. Family League’s 
strong relationships with the city and the state, as well as with nonprofits, foundations, 
universities and businesses, make it an essential organization to bring diverse groups together to 
accomplish change in Baltimore. 

Out of School Time Programs 

The Family League, with support from the City of Baltimore, the State of Maryland, local 
foundations and community partners, has a long history of strong investment in and support of 
quality out of school time programs throughout Baltimore City. Over the past 12 years, more 
than $60 million has been invested to serve an average of over 5,000 youth annually. The logic 
model (see Appendix A) guiding the work of the Family League’s OST investment asserts that 
engaging youth in high quality OST programs will support improved youth outcomes. Target 
outcomes for OST participants include: 

• Improved school attendance and reduced chronic absenteeism 

• Improved school academic performance 

• Reduced incidents of negative behaviors 

• Improved health/fitness 

• Improved attitudes, relationship skills and resiliency/grit 

Through a 3-year grant from the Wallace Foundation, the Family League has been working with 
community partners to strengthen the system supporting OST programs, and they believe that 
positive youth outcomes will result if the OST system provides the following support: 

• OST programs work as part of a Community School strategy that strengthens their 
connection to and impact on school outcomes; 

• Programs adhere to quality standards / best practices and implement core program 
components: academic support, art enrichment, physical fitness/recreation, and the 
serving of nutritious meals daily; 

• OST programs are engaged in an ongoing youth program quality intervention system 
(YPQI) that includes program assessments, planning, and professional development / 
coaching support; 

• OST programs and their Community School partners are supported through a stronger 
data collection / evaluation system that helps them better: drive timely outcome 
improvement efforts; demonstrate investment impact and inform investment decisions. 
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Innovations 

The Family League has embarked on three exciting innovations that they believe will increase 
their impact on positive youth outcomes. 

Community School Engagement Strategy . In FY 1 3 the Family League implemented a new 
strategy that interwove the previously independent network of Community Schools (CS) and 
Out of School Time (OST) Programs. Community Schools are a network of partnerships 
between the school and other community partners that promote student achievement and family 
and community well-being through the intentional coordination of needed resources and 
services. Baltimore City Schools partners with the Family League at each school. Community 
Schools have a full time Community School Coordinator that is funded through Family League. 

For the 201 1-12 school year, there were 20 Community Schools in Baltimore. A number of 
these had funded out of school time programs, but there was not an intentional, facilitated 
partnership between the two initiatives. In the 2012-13 school year, the Family League funded 
38 Community Schools and every Community School that serves elementary and/or middle 
school students (33 of the 38 schools) also had a funded OST program. Additionally, 
Community Schools have a commitment of support from the school principal and are included 
in key school teams, such as the Student Support Team and the School Family Council. As 
part of this aligned strategy, Community Schools Coordinators, in partnership with the schools, 
are able to better support OST programs to enroll students who need and would benefit the most 
from participation, so that OST programs can better support school improvement. 

ExpandED Schools . The Family League also partnered with City Schools to participate in the 
ExpandED Schools National Demonstration Project, led by The After School Corporation 
(TASC). In Baltimore, three ExpandED school sites are in the process of expanding the school 
day by three hours for the whole school population. As a result, all students will receive the 
benefits of expanded learning time in areas of academics, arts, physical fitness and nutrition. 
These schools will have increased scheduling flexibility allowing them to innovate in how they 
use their school facilities and engage support from educators in the community. 

Inclusion . The Family League, along with its partners the Maryland Out of School Time 
Network (MOST) and the Maryland Disability Law Center, are working to support OST 
programs to better include youth with disabilities in their programs. City Schools’ attendance 
data for students receiving special education services show their attendance is lower than that of 
the student body as a whole. To better engage and increase opportunities for academic success, 
recruitment for these students is a growing priority. Previous OST evaluations have shown that 
while OST programs have reached youth who are eligible for Free and Reduced Meals 
(FARMS) at a high rate, they have not been as successful reaching youth who receive special 
education services. In the 2012-13 school year, training was provided to build program 
awareness and capacity to serve these youth, and support resources were made available. For 
2013-14, support will be intensified in 10 pilot community school sites where there is a high 
percentage of youth with disabilities. 
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Methodology 

This report focuses on the OST goals that can be measured with existing administrative data: 

• Program enrollment, 

• School Attendance during 2011-12 and through three quarters of 2012-13, 

• Promotion to the next grade, and 

• Course grades for those students in grades 6 through 9 in 2012-13. 

Research Questions 

The research questions addressed in this study include: 

Demographics 

• Who enrolled in the Family League OST program in 201 1-12? 

• Who regularly attended the program (i.e. met the participation threshold) 
in 2011-12? 

• Who did not attend regularly? 

Student Outcomes 

• Did students who regularly attended a Family League funded OST 
program in 2011-12: 

o Have higher levels of attendance that year? 
o Did new recruits show a change in attendance? 
o Maintain higher rates of attendance in the following year (2012- 
13)? 

o Get promoted to the next grade more than their peers? 
o Earn higher course grades in 20 1 2- 1 3? 

• Does two years of regular OST participation have a greater impact than 
one year? 


Data Sources 

The Family League and Baltimore City Schools provided data for this study. Family League 
OST student data system, Efforts to Outcomes (ETO), provided data for school years 2010-11 
and 201 1-12. The Family League also provided data on number of seats available for each 
program, program length, and focus of services such as academic or activity related. Data from 
the City Schools student management system (SMS) include additional attendance, suspensions 
and academic measures. City Schools also provided report card data for 2012-13, as well as 
Maryland School Assessment (MSA) results for 2010-11 and 2011-12. See Appendix B for 
further information on file development. 
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Inclusion in Outcome Analysis 

Only students meeting the threshold exposure as defined by the Family League staff working 
with the OST programs are included in the outcome analyses. The threshold was determined as 
a level of participation required before staff perceived there might be a change in student 
behavior (see Appendix B for details). Students who met this participation threshold are referred 
to in this report as OST regular attenders. Overall, 2,893 students were OST participants, and of 
these, 2,089 participants (72.2%) met the participation threshold and were included in the 
outcome analysis. On average the students who did not meet the threshold attended OST 
programs for 36.1 days compared to 121.7 days for the regular attenders. 

Propensity Score Matching 

We used propensity score matching to establish comparison groups of students who resembled 
the OST regular attenders. Matched student sets were developed in two ways. All students were 
matched using demographic data, grade in school, 2010-11 attendance, and suspensions. In 
addition, comparison students were selected from the same school as the OST participant; a 
second match selected comparison students from the same grade but selected from any school 
that also offered OST programs and had students who met the OST dosage requirements. 

The rationale for matching students from the same school was to capture “non-observables” 
such as family support of education and values, assuming families in the same schools made 
similar choices for neighborhoods and schools. The across -district match, on the other hand, 
relaxed the same school requirement and thus allowed for a more exact match on the known 
student background data. We used both sets of matches to provide a richer examination of the 
outcome data and a deeper interpretation of the findings. 

The same school match did not produce a comparison group that was equivalent to our OST 
sample in terms of special education and attendance, so to adjust for this, we regressed each of 
the outcome measures on the background covariates in the matching procedure, thus controlling 
for any observable differences that still remained between the two groups. With this additional 
statistical control, any differences that remain statistically significant are better substantiated. 

See Appendix B for full description of propensity score matching procedures and results. 


Definitions 

School Attendance is measured in several different ways: 

Average Daily Attendance (ADA) is defined using MSDE’s methodology as average daily 
attendance, calculated as the number of days attended divided by the number days enrolled in 
City Schools. 
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Chronic Absence is defined by MSDE as students enrolled for at least 90 days who miss more 
than 20 days. In this report, we define chronic absence as missing more than the equivalent of 
one-ninth of days (or 20/180) of days on roll. 

Severe Chronic Absence is defined by MSDE as students missing more than 40 days a school 
year. It is operationalized in this report as missing more than the equivalent of two-ninths (or 
40/180) of days on roll. 

High attendance is defined by MSDE as students who miss fewer than 5 days per school year. It 
is operationalized in this report as having an ADA greater than 97.2% (or 175/180). 


Limitations 

One of the biggest challenges of this report is capturing non-observables such as student and 
family motivation, parent commitment, and support for education — all factors that would affect 
enrolling in an OST program. These students will likely attend more and outperform their peers 
because of their different family and community cultures, and we cannot control for these 
differences. A lottery design would provide an opportunity to control for that but such data are 
not available. 

Propensity score matching partly overcomes these challenges because students are matched on 
available administrative data. Propensity score matching is less robust than a randomized, 
experimental design that would allow for causal inferences. Since the students in the current 
study were not randomly assigned to the program no causal statements about its effects can be 
made. Therefore we are cautious in interpreting our findings. 

Another challenge to our interpretation of results is our inability to know if our comparison 
students participated in other out-of- school-time programs not sponsored by Family League. 
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Findings 

In this section we respond to each of the research questions identified in the methodology. 


Students Enrolled in Family League OST in 2011-12 

In 2011-12, 3,523 students from kindergarten through grade 12 enrolled in 64 Family League 
sponsored OST programs in 48 schools. The most striking demographic feature of the OST 
programs in 2011-12 is the underrepresentation of boys, especially in the middle grades (see 
Table 1). 


OST programs enrolled a higher percentage of students qualifying for FARMS than the district 
as a whole (92.4% compared to 84.5%, respectively). In 35 of the 47 OST programs serving 
single schools, OST programs enrolled more students qualifying for FARMS than the school 
overall (see Appendix C, Table Cl). Only 1 program served multiple schools. 


Table 1 

Baltimore City Family League OST Students 
Enrolled in2011-12by Gender and Grade 


Enrolled OST 
Female Male 


2011-12 

N 

% 

% 

K 

206 

40.8 

59.2 

1 

325 

52.0 

48.0 

2 

364 

53.3 

46.7 

3 

370 

48.6 

51.4 

4 

348 

49.4 

50.6 

5 

401 

58.9 

41.1 

6 

312 

64.1 

35.9 

7 

406 

61.3 

38.7 

8 

296 

68.9 

31.1 

9 

115 

42.6 

57.4 

10 

146 

52.1 

47.9 

11 

142 

52.1 

47.9 

12 

92 

55.4 

44.6 

Total 

3523 

55.0 

45.0 


Source: Family League 2011-12 OST Data and City 
Schools Enrollment and Attendance Data. 


The OST programs were less successful recruiting students receiving special education services. 
The rate of OST students receiving services was 13.3% compared to City Schools’ average of 
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16.5%. Yet, 19 of the 47 OST programs serving single schools recruited a larger percent of 
students receiving special education services than the school’s average (see Appendix C, Table 
Cl). 

Students Regularly Attending in 2011-12 

Outcome analysis was conducted on students who met a threshold of attendance established by 
the Family League to identify the level of exposure to programming sufficient to see a 
difference in behavior. Below we describe those students. 

Gender . The gender difference was smaller for OST regular attenders, with a 48.4% to 51.6% 
male to female ratio. There was a consistent female majority in grades 5 to 8 (see Table 2). 


Table 2 

Baltimore City Family League 201 1-12 OST Regular Attenders by Grade and Gender 


Grade 

OST Regular Attenders 

2011-12 

N 

% Female 

% Male 

K 

159 

42.1 

57.9 

1 

257 

52.1 

47.9 

2 

258 

55.4 

44.6 

3 

275 

50.2 

49.8 

4 

257 

46.3 

53.7 

5 

258 

57.0 

43.0 

6 

121 

57.0 

43.0 

7 

154 

53.2 

46.8 

8 

111 

55.0 

45.0 

9 

54 

44.4 

55.6 

10 

80 

53.8 

46.3 

11 

49 

49.0 

51.0 

12 

56 

46.4 

53.6 

Total 

2089 

51.6 

48.4 


Source: Family League 2011-12 OST Data and City 


Schools Enrollment and Attendance Data. 


Race/Ethnicity . African American students were more likely to be OST regular attenders 
(92.7% compared to 85.9% for City Schools) and Hispanic students were regular attenders in 
OST programs at roughly the same rate as the City Schools population (3.6% compared to 
4.8%). (See Appendix C for full background data on OST participants by level of attendance in 
the programs.) 

Free and Reduced Price Meals . OST regular attenders were more likely to be eligible for 
FARMS than the City Schools average (93.5% compared to 84.5). In 36 of the 47 OST 
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programs serving single schools, a higher percentage of OST regular attenders qualified for 
FARMS than the school overall (see Appendix C, Table Cl). 

Special Education Services . Compared to the district as a whole, OST regular attenders received 
services at a lower rate (11.6% compared to 16.5%). Not surprisingly, more than half, or 26 of 
the 47 OST programs serving single schools, had a lower percentage of OST regular attenders 
receiving special education services than the school overall (see Appendix C, Table Cl). 


Students Who Were Not Regular Attenders 

Students who were not included in outcome analysis because they did not meet the OST 
participation threshold were more likely to be African American or Hispanic and receive special 
education services. They were also more likely to have been chronically absent in school in both 
2010-11 and 2011-12. (See Appendix C, Table C5.) 


Table 3 

Baltimore City Family League 2011-12 OST Students in 2011-12 
Eligible Participants Compared to Regular Attenders by Grade 


Grade 

OST Eligible 
Participants* 

OST Regular 
Attenders 

% OST Regular 
Attenders 

2011-12 

N 

N 

% 

K 

187 

159 

85.0 

1 

309 

257 

83.2 

2 

341 

258 

75.7 

3 

339 

275 

81.1 

4 

325 

257 

79.1 

5 

366 

258 

70.5 

6 

211 

121 

57.3 

7 

241 

154 

63.9 

8 

172 

111 

64.5 

9 

83 

54 

65.1 

10 

119 

80 

67.2 

11 

122 

49 

40.2 

12 

78 

56 

71.8 

Total 

2893 

2089 

72.2 


Source: Family League 2011-12 OST Data and City Schools Enrollment and 
Attendance Data. 

*Excludes students who attended programs providing less than 80 days of service or 
students who did not attend City Schools for full year. 


More middle and high school students did not meet the OST participation threshold than 
elementary students — 78.4% of K-5 enrollees attended OST regularly compared to 61.9% of 
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middle school and 59.5% of high school students (see Table 3). The reasons for not attending 
are not currently known. 

In Table 4 we compare the school attendance profiles in 2010-1 1 and 201 1-12 of students who 
met the OST participation threshold and those who did not meet the threshold. Not surprisingly, 
those who did not attend enough to meet the threshold for inclusion in outcome analyses were 
more likely to have been chronically absent in 2010-11. Elementary students, both regular and 
non-regular OST attenders, had higher school attendance levels in 2011-12, i.e. more high 
attenders and fewer chronically absent students. Secondary students had little change. 


Table 4 

Attendance in 2010-11 and 2011-12 for Regular and Non-Regular OST Attenders 
Who Were in City School in Both 2010-11 and 2011-12 


„ , , OST Regular 

School . _ f 

. i Attenders 

Attendance % 201(M1 201M2 

Non-Regular OST 
2010-11 2011-12 

All Eligible OST 
Enrollees 

2010-11 2011-12 

GradesK-5 

High Attenders 34.4 44.4 

Medium 52.9 48.4 

Chronic Absent 11.6 6.6 

Severe CA 1 . 1 0.6 

(N=1388) 

21.9 32.4 

57.4 53.8 

18.3 12.0 

2.3 1.8 
(N=383) 

31.7 41.8 

53.9 49.6 

13.0 7.8 

1.4 0.8 

(N=1771) 

Grades 6-8 

High Attenders 41.9 46.2 

Medium 49.3 45.6 

Chronic Absent 7.4 6.6 

Severe CA 1.3 1.6 

(N=377) 

37.9 38.4 

46.6 43.1 

12.5 10.8 

3.0 7.8 

(N=232) 

40.4 43.2 

48.3 44.7 

9.4 8.2 

2.0 3.9 

(N=609) 

Grades 9-12 

High Attenders 18.7 16.0 

Medium 47. 1 46.7 

Chronic Absent 24.0 24.0 

Severe CA 10.2 13.3 

(N=225) 

18.1 16.9 

46.9 45.6 

23.8 18.8 

11.3 18.8 

(N=160) 

18.4 16.4 

47.0 46.2 

23.9 21.8 

10.6 15.6 

(N=385) 

All Grades 

High Attenders 34.1 41.5 

Medium 51.1 47.7 

Chronic Absent 12.2 8.6 

Severe CA 2.2 2.2 

(N=1990) 

25.9 31.0 

52.0 48.9 

17.7 13.0 

4.4 7.1 

(N=775) 

31.8 38.6 

51.7 48.0 

13.7 9.8 

2.8 3.6 
(N=2765) 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
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Students Outcomes 

For the remainder of the report, OST regular attenders (students who met the participation 
threshold) are compared with statistically matched groups of students. The comparisons are 
made two ways: 

(1) Same School which compares OST regular attenders to a comparison group drawn only 
from students attending the same school. Matches could not be found for all OST students so 
the number of students in the same school analyses is less than the total number of students, and 

(2) Across Schools which compares OST regular attenders to a comparison group from the 
same grade but drawn from any school across the district that enrolled OST regular attenders. 
All OST students are included. 

As noted earlier, the failure of the Same School matches to attain equivalence on all of the 
matching covariates (particularly 2010-11 attendance) prompted us to perform robustness 
checks on all outcomes; that is, we regressed each of the outcome measures on the background 
covariates in the matching procedure, thus controlling for any differences that still remained 
between the two groups. 


Attendance in 2011-12 

A key outcome for Family League OST programs is school attendance. Comparing to similar 
peers who attended their same school, or attended schools across the district, regular OST 
attenders had significantly higher attendance in 2011-12. We compared to students in the same 
school to better control for non-observables (selection criteria and priorities that led families to 
be in that school) and across the district to provide closer matches on the observable data in our 
data set. 

The within school comparisons include fewer students because the pool for matching students 
was smaller and not all students could be matched. Regular attenders, students who met the 
OST participation threshold, had higher rates of school attendance when compared to similar 
peers from their schools (95.0% compared to 93.0%) and across the district (94.9% compared to 
94.0%) (see Table 5). These differences are not significant among high school students. 

As an additional check, we also controlled for prior academic performance using Maryland 
School Assessments (MSA) scores in English and math from 2010-11. Regular OST attenders 
continued to have a significantly higher level of attendance, whether measured as average daily 
attendance, chronic absence or high attendance (see Appendix D, Table D4). 
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Table 5 

Measures of School Attendance for Students in Kindergarten through Grade 12 
for OST Regular Attenders and Comparable Students in Same School and Across District 



Same School- 

Across Schools 


Regular 

Comparison 

Regular 

Comparison 

Outcomes 2011-12 

Attenders 

Group 

Attenders 

Group 


% 

% 

% 

% 

Grades K-5 





Attendance 

95.7* 

93.8 

95.7* 

94.8 

Chronic Absence 

7.0* 

15.2 

6.9* 

9.9 

Severe Chronic Absent 

0.5* 

3.0 

0.5* 

1.5 

High Attenders 

44.5* 

31.5 

44.3* 

35.7 

(N) 

(1292) 

(1292) 

(1299) 

(1299) 

Grades 6-8 





Attendance 

96.0* 

93.9 

95.7 

94.9 

Chronic Absence 

7.1* 

14.7 

8.3 

11.1 

Severe Chronic Absent 

0.3* 

3.2 

1.4 

2.5 

High Attenders 

47.8* 

34.0 

46.1 

43.9 

(N) 

(312) 

(312) 

(362) 

(362) 

Grades 9-12 





Attendance 

87.1 

84.5 

87.1 

85.6 

Chronic Absence 

40.1 

46.9 

40.1 

40.1 

Severe Chronic Absent 

13.0 

20.4 

13.0 

20.4 

High Attenders 

12.4 

16.7 

12.4* 

22.2 

(N) 

(162) 

(162) 

(162) 

(162) 

Total - All Grades 





Attendance 

95.0* 

93.0 

94.9* 

94.0 

Chronic Absence 

10.0* 

18.0 

10.1* 

12.8 

Severe Chronic Absent 

1.6* 

4.6 

1.8* 

3.3 

High Attenders 

42.1* 

30.6 

41.9 

36.1 

(N) 

(1766) 

(1766) 

(1823) 

(1823) 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 

- Comparison groups within the same school did not attain baseline equivalence in 2010-1 1 attendance. 
* significant at 95% confidence level 


A comparison of school attendance in 2010-11 and 2011-12 by attendance categories (see 
Figure 1) for OST students and comparable peers shows the number of chronically absent OST 
students drops significantly, 3.5 percentage points or from 248 students to 185 compared to a 
0.2 percentage point drop or a reduction from 236 to 233 for comparable peers across the 
district. We compared across the district as it includes a larger sample of students for 
comparison and the two groups better matched on attendance in 2010-11. 
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Figure 1. Number and percent of students by attendance categories in 2010-11 and 2011-12 for 
OST regular attenders and comparable peers across the district. 



Figure 2 displays 2011-12 attendance categories for students who were chronically absent in 
2010-11. The second bar describes their attendance in 2011-12. Almost two-thirds (61.7%) of 
regular attenders, students who met the participation threshold, are no longer chronically absent 
compared to half (51.3%) of their comparable peers, a statistically significant difference. 
Furthermore, 75.8% of these OST students were not chronically absent the following year 
(2012-13, not presented). 


Figure 2. Number and percent of students by attendance categories in 2011-12 for students 

chronically absent in 2010-11 for regular OST attenders and comparable peers across 
the district 
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Attendance of New OST Participants 

Next we examined students who first participated in any OST program in 201 1-12. These new 
recruits had significantly higher school attendance in the year of OST participation (2011-12), 
than comparable peers, whether measured by average daily attendance, chronic absence or rate 
of high attendance (see Table 6). 


Table 6 

Measures of Attendance in 2011-12 for Students in Grades K-12 for Newly Recruited OST Regular 
Attenders and Comparable Students in Same School and Across District 



Same School- 

Across Schools 

Outcomes 

2011-12 

Regular 

Attenders 

% 

Comparison 

Group 

% 

Regular 

Attenders 

% 

Comparison 

Group 

% 

Total - All Grades 
Attendance 

94.2* 

92.0 

94.2* 

92.4 

Chronic Absence 

12.1* 

19.5 

12.2* 

17.9 

Severe Chronic 

2.7* 

7.5 

2.7* 

5.8 

Absence 
High Attenders 

35.2* 

29.6 

35.2 

31.7 

(N) 

(948) 

(948) 

(954) 

(954) 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 

~ Comparison groups within the same school did not attain baseline equivalence in 2010-1 1 attendance. 
* significant at 95% confidence level 


To provide a different perspective we next looked at these students by school attendance 
categories. These newly recruited OST students who met the participation threshold are 
significantly less likely to be chronically absent in 201 1-12 than comparable peers across the 
district, a 5.3 percentage point drop in chronic absence compared to a 0.6 drop for the 
comparison group. (See Figure 3.) 

We compared across the district as it includes a larger sample of students for comparison. This 
change in attendance may reflect a change in participation in OST, or may reflect a change in 
family or life circumstances that made both school attendance and participation in an OST 
program more likely. 
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Figure 3. Number and percent of students by attendance categories in 2010-11 and 2011-12 for 
new recruits who regularly attended OST and comparable peers across the district. 
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In Figure 4 we examine those students who were chronically absent in 2010-1 1. The majority 
of newly recruited OST regular attenders (67.7%) were no longer chronically absent in 2011-12 
while roughly half (48.0%) of the comparable students continued to be chronically absent. 
Additionally, 80.8% of these OST students maintained this level of attendance (not chronically 
absent) into the following year (2012-13). 


Figure 4. Number and percent of students by attendance categories in 201 1-12 for students 
chronically absent in 2010-11 for new recruits who regularly attended OST and 
comparable peers across the district. 
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Attendance in 2012-13 

These same attendance patterns continued into the following year. OST regular attenders, 
students who met the participation threshold, maintained significantly higher levels of school 
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attendance in 2012-13 compared to their peers in the same schools or across the district (see 
Table 7). 


Table 7 

Measures of Attendance in 2012-13 for Students in Kindergarten through Grade 12 


for OST Regular Attenders and Com 

parable Students 

Same School- 

Across Schools 


Regular 

Comparison 

Regular 

Comparison 

Outcomes 2012-13 

Attenders 

Group 

Attenders 

Group 


% 

% 

% 

% 

Grades K-5 





ADA 

95.4* 

93.7 

95.4* 

94.7 

Chronic Absence 

9.5* 

16.1 

9.7 

11.6 

Severe Chronic Absence 

1.0* 

3.2 

1.0* 

2.1 

High Attenders 

46.2* 

35.3 

46.0* 

40.7 

(N) 

(1292) 

(1292) 

(1299) 

(1299) 

Grades 6-8 





ADA 

94.0* 

91.6 

92.9 

91.7 

Chronic Absence 

14.1* 

22.8 

17.4 

19.3 

Severe Chronic Absence 

2.6* 

7.4 

3.6* 

8.0* 

High Attenders 

43.6* 

34.3 

38.7 

36.7 

(N) 

(312) 

(312) 

(362) 

(362) 

Grades 9-12 





ADA 

82.1 

80.7 

82.1 

79.7 

Chronic Absence 

45.7 

51.2 

45.7 

47.5 

Severe Chronic Absence 

25.9 

34.0 

25.9 

28.4 

High Attenders 

18.5 

17.3 

18.5 

19.8 

(N) 

(162) 

(162) 

(162) 

(162) 

Total - All Grades 





ADA 

93.9* 

92.2 

93.7* 

92.8 

Chronic Absence 

13.6* 

20.5 

14.4 

16.3 

Severe Chronic Absence 

3.6* 

6.7 

3.7* 

5.6 

High Attenders 

43.2* 

33.5 

42.1* 

38.0 

(N) 

(1766) 

(1766) 

(1823) 

(1823) 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 

~ Comparison groups within the same school did not attain baseline equivalence in 2010-11 attendance 
* significant at 95% confidence level 


These attendance differences are significant for students in K through grade 8, but not high 
school students. The differences in K through grade 8 were still significant when tested for 
robustness. 
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Of particular note, the higher attendance rates were present among entering 6 th and 9 th graders 
when compared to their peers in the same school. That is, students in 5 th and 8 th grades in 201 1- 
12 showed higher school attendance through the first three quarters of 2012-13 as they were 
transitioning into middle school and high school, respectively. 

However, since we do not know which of the OST students continued to participate in OST 
programs in 2012-13, these results should be viewed with caution. 


Promotion 

These positive trends in attendance appear to translate into higher rates of promotion (see Table 
8). Regular OST attenders were significantly more likely to be promoted compared to peers 
from their schools and across the district. Promotion rates were significantly higher in grades K 
through 8, and high school students had a higher credit accrual rate. Robustness checks 
confirmed significant differences. 


Table 8 

Promotion Rates for OST Regular Attenders in Kindergarten through Grade 12 
and Comparable Students in the Same School and Across the District 


% Promoted 
2012-13 

Same School 

Across Schools 

Regular 

Attenders 

Comparison 

Group 

Regular 

Attenders 

Comparison 

Group 

Elementary 

98.4* 

97.2 

98.4* 

97.1 

Middle 

99.4* 

96.5 

99.2* 

94.8 

High 

87.0* 

78.4 

87.0 

80.9 

All Grades 
(N) 

97.5* 

(1766) 

95.4 

(1766) 

97.5* 

(1823) 

95.2 

(1823) 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data, 
and Quarter 2 report cards. 

~ Comparison groups within the same school did not attain baseline equivalence in 2010-11 attendance 
* significant at 95% confidence level 


2012-13 Course Grades 

For students entering grades 6 through 9 in 2012-13, OST regular attenders, students who met 
the participation threshold, had higher grades mid-year in math courses compared to their peers 
in the same schools but not compared to peers across the district; a robustness check did not 
confirm the significance of the difference in math. Other course grades yielded no significant 
differences. The comparisons controlled for prior academic performance (MSA scores in 
English and math). Sample sizes vary as not every student takes every subject each term. 
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Table 9 

Mean First Semester Course Grades for 201 1-12 OST Regular Attenders 
Entering Grades 6 through 9 in 2012-13 and Comparable Students 



Same School 

Across Schools 


Mean (N) 

Mean (N) 

Mean Grades 

Regular 

Comparison 

Regular 

Comparison 


Attenders 

Group 

Attenders 

Group 

English 

73.8 

73.2 

73.4 

73.0 


(533) 

(530) 

(585) 

(578) 

Math 

72.9* 

71.2 

72.0 

72.9 


(522) 

(522) 

(574) 

(569) 

Social Studies 

73.3 

73.7 

73.2 

74.5 


(436) 

(427) 

(455) 

(456) 

Science 

72.4 

72.5 

72.3 

74.2 


(484) 

(469) 

(526) 

(506) 


Source: Family League 2011-12 OST Data and City Schools Enrollment, Attendance Data, and first 
quarter report cards. 

~ Comparison groups within the same school did not attain baseline equivalence in 2010-1 1 attendance 
* significant at the 95% confidence level 


Students Who Attended Two Years of Family League Sponsored OST 

To determine if continuous participation over two years might have a greater influence on 
students’ performance and behaviors, we identified regular attenders, students who met the 
Family League OST participation threshold in 2010-11 and 2011-12, and compared them to 
students who met the threshold for one year (201 1-12). 

Demographics . A total of 807 students (27.9% of 2011-12 eligible OST participants) regularly 
attended OST programs for two years, 2010-11 and 201 1-12. Two-year OST participants were 
somewhat less likely to be boys, but otherwise resembled OST participants who were regular 
attenders for one year; that is, they had similar levels of FARMS and Special Education (see 
Table 10). 

Attendance. When we controlled for students’ background characteristics and their 2009-10 
school attendance and MSA scores, the two-year OST regular attenders had significantly higher 
levels of attendance in 201 1-12 than the one-year regular attenders; they were not significantly 
less likely to be CA. These differences are not unexpected since the continuity in regular OST 
attendance over two years would suggest higher levels of school attendance. 

Again, we caution against interpreting this as causation because our definition of regular 
attendance assures that OST participants have higher rates of attendance. However, these 
differences did not continue into the following year. The two-year regular attenders did not have 
higher attendance through the third quarter of 2012-13. 
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Table 10 


Demographic Characteristics of 

Students Who Regularly Attended OST for One Year and Two Years 


One- Year 
(N = 1052) 

% 

Two-year 
(N = 807) 

% 

Male 

50.4 

45.4* 

African American 

92.1 

93.6 

Hispanic/Latino 

4.5 

4.1 

FARMS, 2011-12 

93.5 

93.6 

Special Ed, 2011-12 

11.9 

11.3 


Source: Family League 2010-11 and 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
* significant at 95% confidence level 


Promotion . The two-year OST participants were not more likely to be promoted than the one- 
year attenders. 

2012-13 Course Grades . There was no significant difference in the mid-year course marks for 
the one-year and two-year regular participants. 

We do not know which of the OST students continued to participate in OST programs in 2012- 
13, so the lack of differences in attendance and course marks in 2012-13 is difficult to interpret 
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Discussion and Recommendations 

This analysis is dependent on the quality of data from individual programs. Evaluating student 
outcomes is also a challenge because of the self-selection of participants. Students and families 
who opt into these programs are likely different from those who do not. For this reason finding 
comparable students is extremely challenging as these differences due to self-selection are not 
observable, i.e., students’ and parents’ value of education, desire for success, motivation, and 
determination are not measured and included in analyses. Propensity score matching only partly 
overcomes these challenges. For this reason we are cautious in interpreting our findings. 

The Family Feague’s 2011-12 Out of School Time (OST) program enrolled 3,523 students in 
kindergarten through grade 12, in 64 Family Feague sponsored OST programs in 48 schools. 
Over half (57.2%) of the enrolled students were in the elementary grades (Kindergarten through 
grade 5). Enrolled students qualified for Free/Reduced Price Meals (FARMS) at higher rates 
than the district and their school, were less likely to be receiving special education services, 
more likely to be chronically absent, and less often male. 

Outcome analysis was conducted on students who met a threshold of exposure to the program 
as determined by Family Feague. 

Of those eligible, 72.2% attended OST programs enough to be included in outcome analyses. 
This left more than a quarter (27.8%) of students out of the outcome analysis. Those who did 
not meet the participation threshold were more likely to be African American or Hispanic, 
receive special education services, and be in secondary grades. They were also more likely to 
have been chronically absent in the previous year (2010-11). 

The specific reasons for not attending are not currently known and are something OST staff 
should investigate to increase students’ engagement and attendance in the program. 

Specifically, the lower rate of regular OST attendance among students receiving special 
education services is especially problematic as these are students who could most benefit from 
the extra resources and support. 

Our analysis suggests that students in grades K through 8 who attended OST regularly had 
higher rates of school attendance during the 2011-12 school year, and through the first three 
quarters of the next school year than comparable peers who attended the same schools. While 
we cannot attribute this to the OST program, we can say that the evidence is encouraging and 
suggests that schools should urge students to participate in OST programs. The continued higher 
rates of attendance into the next school year may suggest that once established, attendance 
behavior persists into the next school year. Furthermore, targeting students with poorer 
attendance may benefit both the child and the school. 

For newly recruited students who were chronically absent prior to participation in OST, they 
had higher rates of school attendance and fewer students were chronically absent again. That 
may indicate that OST participation supported increased attendance or may reflect changed 
family circumstances resulting in more support for school. These could be finding a job, better 
access to transportation, etc. 


Family League OST 2011-12 


20 


Baltimore Education Research Consortium 


Among entering 6 th and 9 th graders, OST regular attenders had significantly higher attendance 
during the first half of 2012-13 than comparable peers. These are well known difficult transition 
points for students, so seeing higher attendance during the transition is heartening and needs to 
be investigated more fully. 

Not surprisingly, higher school attendance rates for OST participants also appear to translate 
into higher promotion rates — regular OST attenders were significantly more likely to be 
promoted to the next grade than a comparison group of their peers. Of special note, these 
benefits extended into high school where OST regular attenders had a higher credit accrual rate 
than their peers; failure to accrue sufficient credits to advance in grade is a problem throughout 
City Schools high schools. 

We saw little evidence that middle and high school students earned higher grades in academic 
courses the following year if they attended OST programs. This may reflect the fact that 
attendance in OST programs declined after grade 5; thus, we are evaluating effects for a small 
sample of students. 

In sum, although students who participate in OST programs had higher rates of school 
attendance, this analysis cannot make a causal statement that participation increased attendance. 
What the data do show is that compared to non-OST students who are similar in terms of 
background characteristics and previous school attendance, the OST regular attenders had 
higher rates of attendance during that school year, and through the first three quarters of the 
following school year. This evidence is encouraging and suggests that schools should promote 
participation in OST programs. The extra supports conferred by participation in after-school 
programs may enhance academic and social skills as well as physical and emotional health. 
Ultimately such benefits should increase their attachment to school and their academic 
performance. 


Recommendations 

The Family League’s OST programs seek to support youth development for City Schools 
students by offering high quality after-school programs that provide academic enrichment and 
develop positive attitudes, non-cognitive skills, and resiliency. Ultimately, the goal is for OST 
participants to graduate from high school with the skills and behaviors to be successful in 
college and career. Situated in Community Resource Schools, they are able to connect youth 
and families to services and resources to support these goals. Their target populations are at-risk 
students from economically disadvantaged homes, students with special needs, and students 
with a history of chronic absence. 

Data quality issues continue to be a challenge in this work. Family League staff should better 
monitor data collection and the accuracy of pupil IDs during the school year, and as part of that 
process monitor recruitment and the regular attendance of their students. Family League should 
also consider expanding data collection to other areas of students’ lives like health service 
provision and summer school enrollment. 
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The Family League should develop a recruitment plan to increase enrollment paired with 
strategies to improve attendance in OST programs for City Schools students who receive 
special education services, as these students had lower enrollment and attendance rates. Efforts 
should be made to identify barriers to enrollment and attendance to inform program practices. 
These students, who were underrepresented in OST programs in 2010-11 and 2011-12, could 
benefit from the additional supports provided by OST programs to develop both their academic 
and social skills. 

Recruitment efforts should also focus on students with low attendance . Our evaluation suggests 
that OST participation may be valuable for chronically absent students. We are unable to say 
that OST programs cause improved student attendance, but it is reasonable to conclude that 
regular attendance in these programs enhances attachment to school. 

Over a quarter of students (27.8%), enrolled in the program, but did not attend regularly. 
Understanding why these children enrolled, but then chose not to attend will help the Family 
League in recruiting and maintaining high levels of program attendance. These students tended 
to have lower rates of school attendance and to be receiving special education services, and are 
hence part of the target population Family League seeks to serve. 

There should to be move focus on programs aimed at middle and high school students. Middle 
and high school students who enrolled in programs were less likely to attend regularly than 
elementary age students. Middle school is a challenging developmental stage and close attention 
needs to be paid to attendance and academic performance, which could be precursors to later 
dropping out of school (BERC, 2011). Students less engaged with school will only fall further 
behind as they move into high school. There should be special attention paid to programs 
specific to the needs and interests of middle and high school students. 

A key goal of the OST program is improving children’s social and emotional development and 
building resiliency, but no measures have been identified to date to document these changes. 

We recommend that Family League adopt measurement tools that assess socio-emotional 
development , such as the California Healthy Kids Survey, to determine students’ needs and to 
evaluate how participation in OST impacts students’ socio-emotional health. 

As Family League collects measures of the quality of OST programs, future evaluations should 
use these measures to monitor the effectiveness of programs and understand which types of 
programs and resources have an impact on the student populations they serve. This will enable 
OST programs to better recruit and serve City Schools students by providing opportunities to 
develop the academic, social and non-cognitive skills of students at all grade levels. 
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Appendix A: Family League OST Logic Model 
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Appendix B: Methodology 

Data Sources. Data were provided by the Family League student data system, Efforts to Outcomes 
(ETO), for school years 2010-11 and 201 1-12. The Family League also provided data on number 
of seats available for each program, program length, and focus of services such as academic or 
activity related. Students with accurate City Schools pupil ID were merged with data from the City 
Schools student management system (SMS) to include additional attendance, behavior, and 
academic measures. City Schools also provided report card data for 2012-13, as well as Maryland 
School Assessment (MSA) results for 2010-11 and 2011-12. 

Data File Development. The original file from Family League contained records for 6117 
students. We were able to find valid ID matches with the 2011-12 City Schools A-file for 5533 
students (90.5%). A total of 319 students had multiple records, i.e. they attended more than one 
program or attended OST programs in both 2010-11 and 2011-12. Most of the students without a 
valid ID match attended OST programs only in 2010-11 (N=425); 157 of the missing students 
attended OST programs in 2010-11. 

Examining the file, we noted that in some cases there were two different attendance records for the 
same student in the same program in the same year. We made the decision to select the highest 
attendance level for such cases. 

For the year 2012 four programs enrolled fewer than 5 students: 

• CRA02 Open Minds at Patterson Park Library 

• CRA04 Open Minds at Hampstead Hill Academy 

• CRA06 at Patterson Park Public Charter 

• PPE30 High School Volleyball Sports 

These programs are excluded in analyses by program site, but are included in descriptive tables of 
students attending OST programs in 2011-12. None of the students in these four programs met the 
“regular” attender threshold. 

Data for 114 students enrolled at 7 other sites were also excluded because these sites were no 
longer funded by Family League in 2011-12: 

• EBD02 at East Baltimore 

• KBC01 Project Safe Haven 

• LCF05 Powerhouse at Living Classrooms 

• RPC25 at Performing Arts at Cahill 

• UDA01 US Dream Academy 

• YMC06 at Robert Coleman Elementary 

• AA102 at Hampden Family Center 

Since this evaluation pertains to OST participation in 201 1-12, only students who attended OST 
programs in 201 1-12 are included; data from OST participation in 2010-1 1 are only included for 
students who also attended programs the following year. This evaluation only included students in 
grades K- 12 so 21 Pre-K students were excluded. 


Family League OST 2011-12 


25 


Baltimore Education Research Consortium 


Altogether, 3523 students attended Family League sponsored OST programs in 2011-12. 

To be included in the outcome analysis, we included students with specific levels of “dosage.” 
Students are defined as “regular attenders,” if they attend in a given year 80 or more days for 
programs providing 2 and a half hours of service and 60 or more days for programs providing four 
or more hours of service as specified in their contracts with the Family League. In addition, 
students who attended more than one program were classified as “regular attenders” if their total 
number of days of attendance in all OST programs was at least 80 days. 

Most programs provided 2 and a half hours of service; the group of programs providing 4 or more 
hours included: 

• AOCOl at Youth Development Center 

• HAB01 at Collington Square 

• HAB03 at Edgecombe Circle 

• HTA02 Mi Espacio Carrera 

• HTA01 Young Executives Carrera 

• CCY02 at Benjamin Franklin Jr High 

• LAI01 at Loving Arms Solution Center 

• BGC04 at CC Jackson Recreation Ctr 

Family League acknowledged that this threshold was unobtainable for some programs as their 
length was too short. This excluded 16 programs serving 470 students from our analysis. An 
additional 161 students who enrolled after October 1, 2011 or left City Schools before May 1, 

2012 were excluded because they were not enrolled in City Schools long enough to meet the 
threshold. This left a pool of 2,893 participants who could have met the exposure threshold. Of 
these eligible students, 2,089 participants (72.2%) attended OST programs enough to be included 
in the outcome analysis. On average the students who did not meet the threshold attended OST 
programs for 36.1 days compared to 121.7 days for the regular attenders. 

The 16 programs that did not provide at least 80 days of service were: 

• BTP03 at Guilford Elem-Middle 

• BTP05 at Montebello 

• BTP07 at Northeast Middle 

• BTP08 at Hazelwood Elem-Middle 

• CLA03 at Baltimore Talent Development 

• CRA02 Open Minds at Patterson Park Library 

• CRA04 Open Minds at Hampstead Hill Academy 

• CRA06 at Patterson Park Public Charter 

• CRA07 at Highlandtown Elementary 

• PPE10 Middle School Volleyball Sports 

• PPE29 Boys and Girls Soccer Sports 

• PPE30 High School Volleyball Sports 

• UCT01 at Lake Clifton High 

• WAM04 Baltimore Speaks Out at Orleans 
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• WAM05 Baltimore Speaks Out at Herring Run 

• YMC37 at Booker T Washington Middle 

In 2011-12, 2089 students were “regular attenders.” A much smaller group of students (807) 
received a “double dose” of OST services, i.e., they attended programs regularly for two years 
(2010-11 and 2011-12). 

Propensity Score Matching for Comparison Groups 

The final treatment group of regular attenders in 2011-12 used to analyze outcomes in 2011-12 
included 1823 students in grades K-12 (266 students were excluded because they were missing 
2010-11 data. 

We used propensity score matching to establish comparison groups of students who resembled the 
OST treatment groups. Comparison group students were drawn from among all students not in an 
OST program in either 2010-11 or 2011-12 who attended the same schools as the treatment group 
of regular attenders. Thus the comparison group was drawn from among 35,784 City Schools 
students who met the above criteria and had background data available for 2010-11. 

We used the ‘Matchlt’ program in R as a matching algorithm, employing “nearest neighbor 
matching,” choosing the comparison group in two ways. First, we matched exactly on school and 
grade and with additional covariates: gender, Hispanic, African American, age, homeless, receipt 
of FARMS and special education services, prior attendance, and suspension history. A second 
match selected comparison students from the same grade but selected from any school across the 
district that enrolled students from our treatment group. This technique returned for each treatment 
student a single control student who, on average, looked like the treatment student in terms of the 
covariates in the matching model. 

The matching procedure requiring an exact match on both grade and school was unable to match 
57 of the 1823 treatment students, yielding final treatment and comparison groups of 1766 cases 
each. Most of the 57 missing cases were from one school; a high percentage of students in this 
school participated in OST programs so there were few potential comparison group students in this 
school. The second match that matched exactly only on grade was able to match all 1823 cases. 

Comparing results from the two sets of matches (see Table Bl), the exact match on grade only 
produced a comparison group that is not significantly different on any of the matching co variates, 
but the exact match on both grade and school produced a comparison group that is significantly 
different from the treatment group in terms of Special Education and attendance. In other words, 
in some cases there was not a good match available for all cases within a particular school. 
Although the exact match on grade and school produced a somewhat poorer technical match, we 
used results from both sets of matches. The match that constrains on school controls some 
important unknown qualities of the students’ learning environment. So we should consider the 
results from the two different matching procedures with the understanding of the trade-offs 
between the two matching methods. As another check on the robustness of the match, we regressed 
all outcome measures on the original covariates in the matching procedure. If treatment effects are 
still significant this gives us added assurance of the adequacy of the match. 
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Table B1 

Regular Attenders in 2011-12: Comparison of Matched Samples 



Grade Exact Match 

Grade & School Exact Match 

Covariates in Matching 
Model 

OST Non- 
Participants 
(N = 1823) 

OST Regular 
Attenders 
(N = 1823) 

P- 

Value 

OST Non- 
Participants 
(N = 1766) 

OST Regular 
Attenders 
(N = 1766) 

P- 

Value 

Mean 

SE 

Mean 

SE 


Mean 

SE 

Mean 

SE 


Background Controls 











% Male 

47.5 

.012 

47.1 

.012 


48.0 

.012 

46.8 

.012 


% Hispanic/Latino 

4.8 

.005 

4.2 

.005 


3.9 

.005 

4.4 

.005 


% African American 

92.2 

.006 

93.0 

.006 


93.1 

.006 

92.8 

.006 


% Free/Reduced Price Lunch 
Eligibility, 2010-11 

92.6 

.006 

92.9 

.006 


94.1 

.006 

92.8 

.006 


% Special Ed, 2010-11 

11.0 

.007 

10.6 

.007 


12.7 

.008 

10.4 

.007 

* 

% Student Homeless, 2010-1 1 

3.3 

.004 

3.2 

.004 


3.5 

.004 

3.3 

.004 


% Attendance Rate 2010-11 

94.4 

.141 

94.3 

.143 


93.2 

.153 

94.4 

.144 

* 

% Chronic Absence, 201 0-1 1 

12.9 

.008 

13.6 

.008 


18.8 

.009 

13.4 

.008 

* 

%Suspended, 2010-11 

4.9 

.005 

5.3 

.005 


5.9 

.006 

5.0 

.005 


Age on 9/1/2011 

9.8 

.068 

9.8 

.069 


9.8 

.070 

9.7 

.070 














* significant at the 95% confidence level 
~ significant at the 90% confidence level 


Several additional sets of comparison groups were identified to analyze specific outcomes. These 
comparison groups were drawn in a similar manner as outlined above. 

Our analysis of how attendance was impacted by OST participation for a sample of students who 
had no OST participation in 2010-11 included 954 regular attenders who were new recruits to the 
OST program. The exact match on grade and school failed to find matches for 6 students, and it 
provided a somewhat weaker technical match on the covariates (see Table B2). 
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Table B2 

Regular OST Attenders in 2011-12 and No OST in 2010-11 
Compared to Matched Samples 



Grade Exact Match 

Grade & School Exact Match 

Covariates in Matching 
Model 

OST Non- 
Participants 
(N = 954) 

OST 

Participants 
(N = 954) 

P- 

Value 

OST Non- 
Participants 
(N = 948) 

OST 

Participants 
(N = 948) 

P- 

Value 

Mean 

SE 

Mean 

SE 


Mean 

SE 

Mean 

SE 


Background Controls 











Gender 

47.8% 

.016 

49.4% 

.016 


50.5% 

.016 

49.4% 

.016 


Hispanic/Latino 

3.2% 

.006 

3.5% 

.006 


3.2% 

.006 

3.5% 

.006 


African American 

93.7% 

.008 

92.9% 

.008 


93.5% 

.008 

92.8% 

.008 


Free/Reduced Price Lunch 
Eligibility, 2010-11 

90.3% 

.010 

91.6% 

.009 


94.2% 

.008 

91 .7% 

.009 

k 

Special Ed, 2010-11 

8.5% 

.009 

1 1 .2% 

.010 

* 

10.7% 

.010 

1 1 .2% 

.010 


Student Homeless, 201 0-1 1 

4.6% 

.007 

4.2% 

.006 


2.4% 

.005 

4.2% 

.007 

k 

Attendance Rate 201 0-1 1 

92.9 

.243 

93.4 

.218 


92.9 

.237 

93.4 

.219 

+ 

Chronic Absence, 201 0-1 1 

18.6% 

.013 

17.5% 

.012 


19.3% 

.013 

17.4% 

.012 


Suspended, 2010-11 

5.9% 

.008 

5.6% 

.007 


6.0% 

.008 

5.6% 

.007 


Age on 9/1/2011 

9.5 

.107 

9.5 

.107 


9.5 

.107 

9.5 

.107 



* significant at the 95% confidence level 
~ significant at the 90% confidence level 


The comparison groups used to examine academic outcomes in 2012-13 added students’ MSA 
scores in English and math as matching co variates. This reduced the size of the treatment group to 
592 since only students in grades 3-8 in 2010-11 had MSA scores; students with “modified” MSA 
scores were also excluded since their scores were not on the same metric as the students who took 
the “regular” MSA exam (see Table B3). 
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Table B3 

Regular OST Attenders in 2011-12, Grades 5-9 
Compared to Matched Samples (with MSA controls) 


Covariates in Matching 
Model 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 592) 

OST 

Participants 
(N = 592) 

P- 

Value 

OST Non- 
Participants 
(N = 540) 

OST 

Participants 
(N = 540) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Background Controls 
% Male 

% Hispanic/Latino 

% African American 

% Free/Reduced Price Lunch 
Eligibility, 2010-11 

% Special Ed, 2010-11 

% Student Homeless, 2010-11 

% Attendance Rate 2010-11 

% Chronic Absence, 201 0-1 1 

%Suspended, 2010-11 

Age on 9/1/2011 

Reading MSA, 2010-11 

Math MSA, 2010-11 











43.4 

.020 

42.9 

.020 


44.4 

.021 

41.3 

.021 


3.2 

.007 

3.5 

.008 


3.1 

.008 

3.9 

.008 


95.4 

.009 

95.3 

.009 


95.6 

.009 

94.8 

.010 


93.6 

.010 

94.1 

.010 


93.7 

.010 

93.7 

.010 


10.1 

.012 

11.7 

.013 


12.2 

.014 

11.7 

.014 


3.9 

.008 

2.5 

.006 


3.0 

.007 

2.6 

.007 


95.3 

.202 

95.2 

.218 


94.4 

.248 

95.4 

.219 

* 

8.8 

.012 

9.3 

.012 


11.5 

.014 

8.5 

.012 


7.3 

.011 

8.3 

.011 


10.6 

.013 

7.6 

.011 

~ 

12.0 

.055 

12.0 

.057 


11.9 

.059 

11.9 

.060 


397.7 

1.243 

397.8 

1.253 


395.4 

1.341 

398.7 

1.313 

~ 

408.3 

1.388 

406.3 

1.299 


404.0 

1.362 

407.5 

1.366 

~ 












* significant at the 95% confidence level ~ significant at the 90% confidence level 
Note: N’s for Q2 marks are lower than for attendance outcomes (N= 455-585: N= 427-533 


The comparison groups for the “double dose” OST students were drawn in a similar manner as 
outlined above. The 806 students who were high OST attenders in both 2010-11 and 2011-12 were 
matched with 32,596 City Schools students who met the matching criteria. Again the exact match 
on grade and school failed to find matches for 13 students, and it provided a somewhat weaker 
technical match on the co variates (see Table B4). 
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Table B4 

Regular OST Attenders in Both 2010-11 and 2011-12: Comparison of Matched Samples 


Covariates in Matching 
Model 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 806) 

Double Dose 
OST 

(N = 806) 

P- 

Value 

OST Non- 
Participants 
(N = 793) 

Double Dose 
OST 

(N = 793) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Background Controls 
Gender 

Hispanic/Latino 
African American 
Free/Reduced Price Lunch 
Eligibility, 2010-11 
Special Ed, 2010-11 
Student Homeless, 201 0-1 1 
Attendance Rate 201 0-1 1 
Chronic Absence, 201 0-1 1 
Suspended, 2010-11 
Age on 9/1/2011 











44 . 8 % 

.018 

45 . 4 % 

.018 


49 . 7 % 

.018 

45 . 5 % 

.018 

~ 

4 . 2 % 

.007 

4 . 1 % 

.007 


4 . 3 % 

.007 

4 . 2 % 

.007 


93 . 9 % 

.008 

93 . 5 % 

.009 


93 . 7 % 

.009 

93 . 4 % 

.009 


92 . 3 % 

.009 

93 . 2 % 

.009 


94 . 6 % 

.008 

93 . 1 % 

.009 


10 . 5 % 

.011 

10 . 9 % 

.011 


12 . 6 % 

.012 

10 . 6 % 

.011 


3 . 0 % 

.006 

2 . 1 % 

.005 


2 . 5 % 

.006 

2 . 1 % 

.005 


95.2 

.195 

95.3 

.200 


94.4 

.197 

95.4 

.199 

* 

8 . 6 % 

.010 

8 . 9 % 

.010 


12 . 1 % 

.012 

8 . 6 % 

.010 

* 

4 . 7 % 

.007 

4 . 6 % 

.007 


6 . 3 % 

.009 

4 . 3 % 

.007 

~ 

10.2 

.103 

10.2 

.105 


10.2 

.105 

10.2 

.106 













* significant at the 95% confidence level 
~ significant at the 90% confidence level 
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Appendix C: Demographics 


Tabled 

Percent Receiving FARMS and Special Education in 201 1-12 for OST Programs Serving One School: 
OST Participants, OST Regular Attenders, School-Wide Average 


OST Program Serving One 
School 

OST 

% FARMS 

% Special Education 

N 

Regular 

Attenders 

OST 

OST 

Regular 

Attenders 

School 

OST 

OST 

Regular 

Attenders 

School 

ACCE 

88 

54 

69.3 

72.2 

75.8 

21.6 

20.4 

16.3 

Afya Public Charter School 

47 

46 

78.7 

78.3 

86.1 

17.0 

15.2 

23.3 

Armistead Gardens School #243 

43 

40 

100.0 

100.0 

92.6 

11.6 

7.5 

15.6 

Baltimore Talent Development 
High School 

8 

5 

87.5 

80.0 

89.5 

12.5 

20.0 

16.4 

Barclay Elementary/Middle 

103 

88 

97.1 

96.6 

93.7 

12.6 

14.8 

11.3 

Bay Brook Elementary/Middle 

200 

163 

98.5 

98.2 

95.0 

13.5 

12.9 

14.8 

Benjamin Franklin High School @ 
Masonville Cove 

61 

34 

91.8 

88.2 

87.3 

23.0 

23.5 

23.5 

Bluford Drew Jemison Stem 
Academy 

16 

16 

87.5 

87.5 

83.0 

25.0 

25.0 

13.9 

Calvin Rodwell Elementary 

142 

133 

91.5 

91.0 

88.1 

5.6 

4.5 

10.2 

Carver Vocational Technical High 

57 

34 

84.2 

85.3 

80.5 

15.8 

17.6 

11.9 

Collington Square 

142 

123 

98.6 

98.4 

95.0 

13.4 

13.0 

15.6 

Commodore John Rodgers 
Elementary / Middle 

59 

55 

98.3 

98.2 

95.0 

15.3 

14.5 

15.9 

Dickey Hill Elementary/Middle 
School 

29 

29 

96.6 

96.6 

93.1 

6.9 

6.9 

11.3 

Dr. Bernard Harris Sr. Elementary 
School 

138 

121 

100.0 

100.0 

95.0 

7.3 

7.4 

12.2 

Dr. Martin Luther King 
Elementary/Middle School 

89 

79 

92.1 

92.4 

95.0 

20.2 

21.5 

19.5 

Dr. Rayner Browne 

136 

119 

100.0 

100.0 

95.0 

6.6 

5.0 

9.5 

Edgewood Elementary School 

102 

92 

95.1 

94.6 

94.5 

10.8 

9.8 

17.2 

Franklin Square Elementary Middle 

10 

10 

100.0 

100.0 

95.0 

10.0 

10.0 

15.7 

Frederick Elementary School 

125 

100 

99.2 

99.0 

95.0 

17.6 

16.0 

13.9 

Furman L. Templeton Elementary 

19 

17 

100.0 

100.0 

95.0 

31.6 

29.4 

13.9 

George Washington Elementary 

97 

80 

94.8 

95.0 

95.0 

15.5 

16.3 

16.4 

Guilford ES/MS 

20 

18 

100.0 

100.0 

94.1 

15.0 

16.7 

14.4 

Hampstead Hill Academy 

3 

3 

100.0 

100.0 

79.8 

0.0 

0.0 

7.9 

Harlem Park Elementary/Middle 
School 

29 

26 

93.1 

92.3 

95.0 

24.1 

23.1 

16.4 

Hazelwood ES/MS 

23 

21 

82.6 

81.0 

85.6 

8.7 

9.5 

24.4 

Heritage High School 

52 

31 

88.5 

87.1 

86.3 

21.2 

19.4 

22.3 

Highlandtown Elementary/Middle 
School 

10 

7 

100.0 

100.0 

94.0 

10.0 

0.0 

13.2 


continued 
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Table Cl - continued 

Percent Receiving FARMS and Special Education in 2011-12 for OST Programs Serving One School: 
OST Participants, OST Regular Attenders, School-Wide Average 


OST Program Serving One 
School 

OST 

% FARMS 

% Special Education 

N 

Regular 

Attenders 

OST 

OST 

Regular 

Attenders 

School 

OST 

OST 

Regular 

Attenders 

School 

Hilton Elementary School 

124 

121 

91.9 

91.7 

91.3 

7.3 

7.4 

16.4 

Inner Harbor East Academy for 
Young Scholars 

118 

115 

86.4 

87.0 

87.6 

9.3 

9.6 

7.5 

Liberty Elementary 

21 

15 

95.2 

100.0 

92.9 

23.8 

26.7 

20.4 

Lockerman Bundy Elementary 

47 

46 

95.7 

95.7 

95.0 

19.2 

17.4 

9.5 

Margaret Brent 

Elementary/Middle School #53 

50 

47 

92.0 

91.5 

90.0 

14.0 

14.9 

15.9 

Middle School for the Arts at 
Booker T. Washington 

44 

36 

95.5 

97.2 

95.0 

15.9 

16.7 

26.4 

Montebello Elementary/Middle 

119 

102 

92.4 

93.1 

92.5 

10.9 

10.8 

15.9 

Morrell Park Elementary/Middle 

48 

41 

93.8 

92.7 

86.0 

18.8 

17.1 

14.0 

National Academy Foundation 

41 

33 

90.2 

87.9 

84.8 

12.2 

6.1 

17.4 

New Era Academy 

99 

81 

87.9 

86.4 

82.3 

9.1 

8.6 

16.4 

Northeast ES/MS 

31 

29 

83.9 

82.8 

93.6 

16.1 

17.2 

19.6 

Patterson High 

84 

56 

92.9 

94.6 

80.0 

16.7 

17.9 

16.4 

Patterson Park Public Charter 
School 

7 

6 

100.0 

100.0 

81.8 

0.0 

0.0 

13.1 

Pimlico Elementary/Middle School 

31 

27 

96.8 

100.0 

95.0 

29.0 

34.6 

16.6 

Southwest Academy 

33 

26 

87.9 

92.0 

86.7 

18.2 

12.0 

16.7 

Tench Tilghman 

128 

117 

99.2 

99.1 

95.0 

14.8 

14.5 

12.6 

W.E.B. DuBoisHigh 

72 

50 

93.1 

94.0 

78.8 

23.6 

18.0 

26.0 

Waverly EMS 

45 

42 

91.1 

90.5 

93.5 

15.6 

16.7 

16.2 

William Paca Elementary 

73 

68 

98.6 

98.5 

95.0 

19.2 

19.1 

13.3 

Wolfe Street Academy 

20 

20 

95.0 

95.0 

95.0 

20.0 

20.0 

15.8 

Total 

3117 

2649 

93.3 

85.0 

90.2 

14.9 

13.2 

15.8 


Family League OST 2011-12 


33 


Baltimore Education Research Consortium 


Table C2 

African American, Hispanic, Limited English, 2011-12 


Grade 2011-12 

Enrolled OST Program in 2011-12 

Attended OST 
Program Regularly (80/60 

Days) 

N 

AA 

% 

Hispanic 

% 

LEP 

% 

N 

AA 

% 

Hispanic 

% 

LEP 

% 










Kind 

206 

93.2 

3.4 

1.0 

159 

95.0 

2.5 

0.6 

1 

325 

91.7 

3.1 

3.4 

257 

92.6 

3.1 

4.3 

2 

364 

89.0 

3.6 

3.6 

258 

89.9 

4.7 

4.3 

3 

370 

90.3 

5.4 

3.2 

275 

93.1 

5.1 

2.5 

4 

348 

92.2 

3.4 

2.3 

257 

93.4 

3.1 

1.9 

5 

401 

92.3 

3.7 

1.0 

258 

94.6 

2.3 

1.2 

6 

312 

86.9 

6.7 

4.2 

121 

91.7 

8.3 

5.0 

7 

406 

87.4 

5.4 

1 

154 

97.4 

1.3 

0.6 

8 

296 

88.9 

6.1 

2.0 

111 

95.5 

3.6 

2.7 

9 

115 

83.5 

12.2 

8.7 

54 

87.0 

13.0 

11.1 

10 

146 

86.3 

8.9 

8.2 

80 

80.0 

12.5 

13.8 

11 

142 

88.7 

6.3 

3.5 

49 

89.8 

8.2 

6.1 

12 

92 

92.4 

2.2 

5.4 

56 

94.6 

3.6 

7.1 










Total 

3523 

89.7 

5.0 

3.1 

2089 

92.7 

4.4 

3.4 










BCPS Total 

87,038 

85.9% 

4.8% 

3.4% 















Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
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Table C3 

% Eligibility for FARMS and Special Education, % Homeless, 2011-12 


Grade 2011- 
12 

Enrolled OST Program in 2011-12 

Attended 

Program Regularly (80/60 Days) 

N 

FARMS 

% 

Homeless 

% 

Spec Ed 

% 

N 

FARMS 

% 

Homeless 

% 

Spec Ed 

% 










Kind 

206 

95.6 

8.3 

7.8 

159 

95.0 

5.7 

6.9 

1 

325 

96.6 

4.0 

8.3 

257 

97.3 

3.1 

5.4 

2 

364 

95.3 

3.8 

11.8 

258 

95.0 

2.7 

11.2 

3 

370 

95.1 

2.7 

13.2 

275 

93.5 

1.8 

10.9 

4 

348 

95.4 

2.9 

12.9 

257 

94.9 

1.9 

11.3 

5 

401 

94.5 

2.5 

15.7 

258 

94.2 

2.7 

14.0 

6 

312 

93.9 

1.9 

13.5 

121 

95.0 

1.7 

14.0 

7 

406 

90.6 

2.5 

.3 

154 

92.9 

1.9 

12.3 

8 

296 

85.1 

2.7 

12.5 

111 

87.4 

3.6 

12.6 

9 

115 

86.1 

3.5 

21.7 

54 

92.6 

3.7 

20.4 

10 

146 

85.6 

0.0 

17.8 

80 

81.3 

0.0 

16.3 

11 

142 

83.1 

2.1 

23.2 

49 

95.9 

4.1 

20.4 

12 

92 

84.8 

4.3 

16.3 

56 

82.1 

1.8 

17.9 










Total 

3523 

92.4 

3.1 

13.3 

2089 

93.5 

2.6 

11.6 










BCPS Total 

87,038 

84.5% 

2.8% 

16.5% 















Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
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Table C4 

Attendance 2011-12 


Grade 2011-12 

Enrolled OST Program 

n 2011-12 

Attended Program Regularly 

N 

Average Daily 
Attendance 

Chronic 

Absence 

N 

Average Daily 
Attendance 

Chronic 

Absence 

% 

% 

% 

% 








Kind 

206 

94.3 

12.6 

159 

94.9 

8.2 

1 

325 

94.6 

12.3 

257 

95.0 

10.5 

2 

364 

94.5 

11.5 

258 

95.5 

7.4 

3 

370 

95.2 

10.5 

275 

95.9 

6.5 

4 

348 

95.3 

8.9 

257 

95.9 

7.0 

5 

401 

95.8 

7.2 

258 

96.2 

5.8 

6 

312 

94.4 

10.9 

121 

96.0 

5.0 

7 

406 

94.1 

12.6 

154 

95.4 

9.7 

8 

296 

94.1 

14.5 

111 

95.8 

9.9 

9 

115 

81.5 

45.2 

54 

83.0 

50.0 

10 

146 

85.4 

41.8 

80 

89.7 

32.5 

11 

142 

85.3 

38.0 

49 

86.0 

42.9 

12 

92 

89.0 

31.5 

56 

90.8 

26.8 








Total 

3523 

93.4 

15.1 

2089 

94.7 

11.1 

BCPS Total 

87,038 

89.8 

24.3 












Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
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Table C5 

Demographics and Attendance of OST Participants by Level of Attendance in Program 



Less Regular OST 
Attendance 
(N = 804) 

Attended Program 
Regularly (80/60 Days) 
(N = 2089) 

P-Value 

Mean 

SE 

Mean 

SE 

Gender-Male 

48.5% 

.018 

48.4% 

.011 

.98 

African American 

86.3% 

.012 

92.7% 

.006 

.00 

Hispanic/Latino 

6.3% 

.009 

4.4% 

.004 

.04 

Free/Reduced Price Lunch 
Eligibility, 2011-12 

93.9% 

.008 

93.5% 

.005 

.68 

Special Ed 

1 7.5% 

.013 

1 1 .6% 

.007 

.00 

Limited English Proficiency 

2.9% 

.006 

3.5% 

.004 

.43 

Student Homeless 

3.7% 

.007 

2.6% 

.004 

.15 

Attendance Rate 201 0-1 1 

92.2% 

.327 

94.1% 

.145 

.00 

Chronic Absence, 201 0-1 1 

20.6% 

.021 

1 2.7% 

.009 

.00 

Attendance Rate 2011-12 

91 .7% 

.414 

94.7% 

.146 

.00 

Chronic Absence, 201 1 -1 2 

19.9% 

.014 

11.1% 

.007 

.00 

High Attenders, 2011-12 

31 .3% 

.016 

41 .0% 

.011 

.00 

Mean Days Attended OST 

36.1 

.798 

121.7 

.587 

.00 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 


Table C6 

Two-Year Regular OST Attenders by Grade 


Grade 

2011-12 

OST 

Participants 

Two-Year Regular 
Attenders 

N 

N 

% 

K 

187 

10 

5.3 

1 

309 

102 

33.0 

2 

341 

97 

28.4 

3 

339 

115 

33.9 

4 

325 

121 

37.2 

5 

366 

97 

26.5 

6 

211 

51 

24.2 

7 

241 

82 

34.0 

8 

172 

54 

31.4 

9 

83 

14 

16.9 

10 

119 

25 

21.0 

11 

122 

15 

12.3 

12 

78 

24 

30.8 

Total 

2893 

807 

27.9 


Source: Family League 2011-12 OST Data and City Schools Enrollment and Attendance Data. 
^Excludes students who attended programs providing less than 80 days of 
service or students who did not attend City Schools for full year. 
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Appendix D: 2011-12 and 2012-13 Outcome Tables 


Table D1 

Attendance in 2011-12 for Students in Kindergarten through Grade 12 
For OST Regular Attenders and Comparison Group 


Outcomes 2011-12 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 1823) 

OST 

Participants 
(N = 1823) 

P- 

Value 

OST Non- 
Participants 
(N = 1766) 

OST 

Participants 
(N = 1766) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

% Attendance Rate 2011-12 

94.0 

.188 

94.9 

.150 

k 

93.0 

.202 

95.0 

.152 

* 

% Chronic Absence, 2011-12 

12.8 

.008 

10.1 

.007 

k 

18.0 

.009 

10.0 

.007 

* 

% Severe Chronic Absence 

3.3 

.004 

1.8 

.003 

k 

4.6 

.005 

1.6 

.003 

k 

% Hi Attenders (<5 days) 

36.1 

.011 

41.9 

.012 


30.6 

.011 

42.1 

.012 

* 


* significant at the 95% confidence level 


~ significant at the 90% confidence level 

Table D2 

Attendance in 2011-12 for Newly Recruited Regular OST Attenders 
in 2011-12 (No OST in 2010-11) Compared to Matched Samples 


Outcomes 2011-12 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 954) 

OST 

Participants 
(N = 954) 

P- 

Value 

OST Non- 
Participants 
(N = 948) 

OST 

Participants 
(N = 948) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Attendance Rate 2011-12 

92.4 

.349 

94.2 

.220 

* 

92.0 

.362 

94.2 

.222 

k 

Chronic Absence, 201 1 -1 2 

17.9% 

.012 

12.2% 

.011 

* 

19.5% 

.013 

12.1% 

.011 

k 

Severe Chronic Absence 

5.8% 

.008 

2.7% 

.005 

* 

7.5% 

.009 

2.7% 

.005 

k 

Hi Attenders (<5 days) 

31.7% 

.015 

35.2% 

.015 

+ 

29.6% 

.015 

35.2% 

.016 

k 


* significant at the 95% confidence level 
~ significant at the 90% confidence level 
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Table D3 

Attendance in First Half of 2012-13 for Students in Kindergarten through Grade 12 
For OST Regular Attenders and Comparison Group 


Outcomes 2012-13 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 1823) 

OST 

Participants 
(N = 1823) 

P- 

Value 

OST Non- 
Participants 
(N = 1766) 

OST 

Participants 
(N = 1766) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

% Attendance Rate 

92.8 

.264 

93.7 

.232 

* 

92.2 

.248 

93.9 

.226 

ie 

% Chronic Absence 

16.3 

.009 

14.4 

.008 


20.5 

.010 

13.6 

.008 

ie 

% Severe Chronic Absence 

5.6 

.005 

3.7 

.004 

* 

6.7 

.006 

3.6 

.004 

* 

% Hi Attenders (<5 days) 

38.0 

.011 

42.1 

.012 

ie 

33.5 

.011 

43.2 

.012 

* 


* significant at the 95% confidence level 
- significant at the 90% confidence level 


Table D4 

2011-12 and 2012-13 Outcomes for Regular OST Attenders in 2011-12, Grades 5-9 

Compared to Matched Samples 


Outcomes 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 592) 

OST 

Participants 
(N = 592) 

P- 

Value 

OST Non- 
Participants 
(N = 540) 

OST 

Participants 
(N = 540) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Outcomes 2011-12 











% Attendance Rate 2011-12 

94.7 

.301 

95.4 

.249 

- 

93.8 

.331 

95.6 

.255 

* 

% Chronic Absence, 2011-12 

11.0 

.013 

9.0 

.012 


13.7 

.015 

8.3 

.012 

* 

% Severe Chronic Absence 

2.4 

.006 

1.7 

.005 


4.3 

.009 

1.1 

.005 

* 

% Hi Attenders (<5 days) 

40.5 

.020 

46.6 

.021 

ie 

33.5 

.020 

48.0 

.022 

* 

Outcomes 2012-13 











% Promoted 

96.3 

.008 

98.1 

.006 

~ 

95.4 

.009 

98.1 

.006 

* 

% Attendance Rate 2010-11 

93.0 

.404 

93.3 

.468 


92.4 

.429 

94.0 

.439 

* 

% Chronic Absence, 2010-1 1 

19.4 

.016 

16.7 

.015 


19.4 

.017 

14.4 

.015 

* 

% Severe Chronic Absence 

6.1 

.010 

3.7 

.008 

~ 

7.0 

.011 

3.0 

.007 

* 

% Hi Attenders (<5 days) 

42.1 

.020 

43.1 

.020 


36.1 

.021 

46.3 

.021 

* 

Q2 Math Mark 

72.9 

.549 

72.0 

.555 


71.2 

.584 

72.9 

.571 

* 

Q2 English Mark 

73.0 

.541 

73.4 

.538 


73.2 

.558 

73.8 

.560 


Q2 Social Studies Mark 

74.5 

.600 

73.2 

.632 


73.7 

.656 

73.3 

.646 


Q2 Science Mark 

74.2 

.581 

72.3 

.551 

ie 

72.5 

.594 

72.4 

.574 














* significant at the 95% confidence level ~ significant at the 90% confidence level 
Note: N’s for Q2 marks are lower than for attendance outcomes (N= 455-585; N= 427-533) 
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Table D5 

Attendance in 2011-12 for Students in Kindergarten through Grade 12 
for OST Regular Attenders in both 2010-11 and 2011-12 and Comparison Group 


Outcomes 2011-12 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 806) 

OST 

Participants 
(N = 806) 

P- 

Value 

OST Non- 
Participants 
(N = 793) 

OST 

Participants 
(N = 793) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Attendance Rate 2011-12 

94 6 

.275 

95.6 

.225 

* 

93.4 

.333 

95.6 

.225 

★ 

Chronic Absence, 201 1 -1 2 

10.3% 

.011 

8.9% 

.010 


14.4% 

.012 

8.8% 

.010 

* 

Severe Chronic Absence 

3.1% 

.006 

1 .6% 

.004 

* 

4.5% 

.007 

1 .4% 

.004 

* 

Hi Attenders (<5 days) 

43.1% 

.017 

50.0% 

.018 

* 

33.9% 

.017 

50.4% 

.018 

* 


* significant at the 95% confidence level 
- significant at the 90% confidence level 


Table D6 

2011-12 and 2012-13 Outcomes for Regular OST Attenders in both 2010-11 and 2011-12 

Compared to Matched Samples 


Outcomes 

Grade Exact Match 

Grade & School Exact Match 

OST Non- 
Participants 
(N = 737) 

OST 

Participants 
(N = 737) 

P- 

Value 

OST Non- 
Participants 
(N = 719) 

OST 

Participants 
(N = 719) 

P- 

Value 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Mean 

SE 

Outcomes 2011-12 











% Attendance Rate 2011-12 

95.0 

.247 

95.7 

.230 

★ 

93.9 

.302 

95.7 

.231 

* 

% Chronic Absence, 2011-12 

10.6 

.011 

8.5 

.010 


13.5 

.013 

8.5 

.010 

* 

% Severe Chronic Absence 

2.4 

.006 

1.4 

.004 


3.5 

.007 

1.1 

.004 

* 

% Hi Attenders (<5 days) 

43.1 

.018 

49.9 

.018 


35.5 

.018 

50.3 

.019 

* 

Outcomes 2012-13 











% Promoted 

97.0 

.006 

97.8 

.005 


95.7 

.008 

97.9 

.005 

* 

% Attendance Rate 2010-11 

93.9 

.324 

94.3 

.355 


92.6 

.426 

94.4 

.362 

* 

% Chronic Absence, 201 0-1 1 

14.5 

.013 

12.6 

.012 


15.7 

.014 

12.2 

.012 

~ 

% Severe Chronic Absence 

4.6 

.008 

3.4 

.007 


5.0 

.008 

3.3 

.007 


% Hi Attenders (<5 days) 

44.0 

.018 

47.9 

.018 


36.0 

.018 

48.8 

.019 

* 


* significant at the 95% confidence level 
~ significant at the 90% confidence level 
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